Marios Richards' Data Science Blog

Mark Pack's Pollbase Analysis

Visualisation of Mark Pack's Pollbase Dataset¶

I picked up the excel sheet from Mark Pack's blog Processed it with a script in this notebook That script creates this flat csv output

This notebook is primarily proof of concept for a workflow that lets me publish Jupyter Notebooks with functional javascript widgets (i.e. please play with the buttons to the right of charts!).

However, I will come back to this and try out some timeseries analysis techniques.

Toggle code visiblity on/off with the button below

In [75]:

from IPython.display import display
from IPython.display import HTML
import IPython.core.display as di # Example: di.display_html('<h3>%s:</h3>' % str, raw=True)

# This line will hide code by default when the notebook is exported as HTML
di.display_html('<script>jQuery(function() {if (jQuery("body.notebook_app").length == 0) { jQuery(".input_area").toggle(); jQuery(".prompt").toggle();}});</script>', raw=True)

# This line will add a button to toggle visibility of code blocks, for use with the HTML export version
di.display_html('''<button onclick="jQuery('.input_area').toggle(); jQuery('.prompt').toggle();">Toggle code</button>''', raw=True)

In [ ]:

In [76]:

# %matplotlib inline
import numpy as np, pandas as pd, matplotlib.pyplot as plt, seaborn as sns
import pickle, os, gc, re

from utility import *
BES_data_folder = '../BES_analysis_data/'
Pollbase_folder = create_subdir(BES_data_folder,"Pollbase")

In [77]:

from bokeh.models import ColumnDataSource
from bokeh.plotting import figure, show,  output_notebook
from bokeh.layouts import column, row
from bokeh.embed import components

In [78]:

output_notebook()

Loading BokehJS ...

In [ ]:

In [79]:

df = pd.read_csv(Pollbase_folder+"MarkPackPollBase_flat_v6.csv",
                 parse_dates=['Fieldwork_Start_Date', 'Fieldwork_Finish_Date',
       'Fieldwork_Midpoint_Date','Fieldwork_Start_check_date'],index_col="Unnamed: 0")

In [ ]:

Simple set of linked charts of Con/Lab/LD voteshare

Controls on the right let you pan, box zoom, box select, mouse wheel zoom, or reset the chart

Charts are all linked so that selection/movement is mirrored

In [80]:

source = ColumnDataSource(df)

p1 = figure(x_axis_type="datetime",x_axis_label = "Fieldwork_Midpoint_Date",y_axis_label="%",
            plot_width=600, plot_height=280, title="Conservative Voting Intention",
            tools='box_select,pan,wheel_zoom,box_zoom,reset', active_drag="box_select")
p1.scatter('Fieldwork_Midpoint_Date', 'Con', source=normal_poll_source, color='blue')

p2 = figure(x_axis_type="datetime",x_axis_label = "Fieldwork_Midpoint_Date",y_axis_label="%",
            plot_width=600, plot_height=280, title="Labour Voting Intention",
            tools='box_select,pan,wheel_zoom,box_zoom,reset', active_drag="box_select")
p2.scatter('Fieldwork_Midpoint_Date', 'Lab', source=normal_poll_source, color='red')

p3 = figure(x_axis_type="datetime",x_axis_label = "Fieldwork_Midpoint_Date",y_axis_label="%",
            plot_width=600, plot_height=280, title="LibDem Voting Intention",
            tools='box_select,pan,wheel_zoom,box_zoom,reset', active_drag="box_select")
p3.scatter('Fieldwork_Midpoint_Date', 'LD', source=normal_poll_source, color='orange')

# Link the x_range of p2 to p1: p2.x_range
p2.x_range = p1.x_range

# Link the y_range of p2 to p1: p2.y_range
p2.y_range = p1.y_range

# Link the x_range of p3 to p1: p3.x_range
p3.x_range = p1.x_range

# Link the y_range of p3 to p1: p3.y_range
p3.y_range = p1.y_range

layout = column(p1,p2,p3)
show(layout)

In [81]:

# df["Poll_Type"].value_counts()

Full set of linked charts for all parties ever appearing in these national polls (ordered by how many polls they participate in)

Controls on the right let you pan, box zoom, box select, mouse wheel zoom, or reset the chart

Charts are all linked so that selection/movement is mirrored but only in the x-axis (not helpful to zoom in a major party y axis while you want to also look at minor parties)

Polls now separated into Normal Polls (circles), General Election Results (big diamonds) and Exit Polls (big X's)

In [82]:

normal_poll_source = ColumnDataSource( df[df["Poll_Type"]=="Normal Poll"] )
result_source = ColumnDataSource( df[df["Poll_Type"]=="Result"] )
exit_poll_source = ColumnDataSource( df[df["Poll_Type"]=="Exit Poll"] )

def VI_time_series(title,col,colour):
    p = figure(x_axis_type="datetime",x_axis_label = "Fieldwork_Midpoint_Date",y_axis_label="%",
                plot_width=600, plot_height=280, title=title,
                tools='box_select,pan,wheel_zoom,box_zoom,reset', active_drag="box_select")
    p.scatter('Fieldwork_Midpoint_Date', col, source=normal_poll_source, color=colour)
    p.diamond('Fieldwork_Midpoint_Date', col, source=result_source, color=colour, size=40,fill_alpha=0.3,line_alpha=0.0)
    p.x('Fieldwork_Midpoint_Date', col, source=exit_poll_source, color=colour, size=40,fill_alpha=0.3,line_alpha=0.3)    
    
    return p

# Conservatives
p1 = VI_time_series(title="Conservative (Voting Intention)",col="Con",colour='blue')

# Labour
p2 = VI_time_series(title="Labour (Voting Intention)",col="Lab",colour='red')

# Con-Lab
p3 = VI_time_series(title="Conservative lead over Labour (Voting Intention)",col="Con lead",colour='pink')

# LibDems
p4 = VI_time_series(title="LibDem (Voting Intention)",col="LD",colour='orange')

# UKIP
p5 = VI_time_series(title="UKIP (Voting Intention)",col="UKIP",colour='purple')

# Green
p6 = VI_time_series(title="Green (Voting Intention)",col="Green",colour='green')

# BNP
p7 = VI_time_series(title="BNP (Voting Intention)",col="BNP",colour='black')

# SDP
p8 = VI_time_series(title="SDP (Voting Intention)",col="SDP",colour='yellow')

# BXP
p9 = VI_time_series(title="BXP (Voting Intention)",col="BXP",colour='cyan')

# TIG/CUK
p10 = VI_time_series(title="TIG/CUK (Voting Intention)",col="TIG/CUK",colour='grey')

# TIG/CUK
p11 = VI_time_series(title="Referendum (Voting Intention)",col="Referendum",colour='purple')

# Link the x/y ranges
# okay - not the y-range, that makes LD invisible!
# p2.y_range = p1.y_range
# p3.y_range = p1.y_range

p2.x_range = p1.x_range
p3.x_range = p1.x_range
figs = [p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11]
for fig in figs[1:]:
    fig.x_range = p1.x_range

layout = column(figs)
show(layout)

TODO:

short period of Male/Female breakdown - 05-10, 84 rows

Leader versions

Govt good/bad/net

Best PM

Predicted Party share - just 25 rows! - compare to polling/results?

Note: Con-Lab lead can be filled in for all values automatically (e.g. exit poll/result entries as well as normal polls)

In [83]:

# df.columns

In [ ]:

links

social